performance measurement
Performance Measurements in the AI-Centric Computing Continuum Systems
Donta, Praveen Kumar, Zhang, Qiyang, Dustdar, Schahram
Over the Eight decades, computing paradigms have shifted from large, centralized systems to compact, distributed architectures, leading to the rise of the Distributed Computing Continuum (DCC). In this model, multiple layers such as cloud, edge, Internet of Things (IoT), and mobile platforms work together to support a wide range of applications. Recently, the emergence of Generative AI and large language models has further intensified the demand for computational resources across this continuum. Although traditional performance metrics have provided a solid foundation, they need to be revisited and expanded to keep pace with changing computational demands and application requirements. Accurate performance measurements benefit both system designers and users by supporting improvements in efficiency and promoting alignment with system goals. In this context, we review commonly used metrics in DCC and IoT environments. We also discuss emerging performance dimensions that address evolving computing needs, such as sustainability, energy efficiency, and system observability. We also outline criteria and considerations for selecting appropriate metrics, aiming to inspire future research and development in this critical area.
- Europe > Sweden > Stockholm > Stockholm (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Europe > Spain (0.04)
- (3 more...)
- Research Report (0.40)
- Overview (0.34)
- Energy (1.00)
- Information Technology > Smart Houses & Appliances (0.49)
- Information Technology > Services (0.48)
- Information Technology > Communications > Networks (1.00)
- Information Technology > Artificial Intelligence > Natural Language (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning (0.93)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning > Generative AI (0.35)
Scalable Principal-Agent Contract Design via Gradient-Based Optimization
Galanti, Tomer, Bookseller, Aarya, Ray, Korok
We study a bilevel \emph{max-max} optimization framework for principal-agent contract design, in which a principal chooses incentives to maximize utility while anticipating the agent's best response. This problem, central to moral hazard and contract theory, underlies applications ranging from market design to delegated portfolio management, hedge fund fee structures, and executive compensation. While linear-quadratic models such as Holmstr"om-Milgrom admit closed-form solutions, realistic environments with nonlinear utilities, stochastic dynamics, or high-dimensional actions generally do not. We introduce a generic algorithmic framework that removes this reliance on closed forms. Our method adapts modern machine learning techniques for bilevel optimization -- using implicit differentiation with conjugate gradients (CG) -- to compute hypergradients efficiently through Hessian-vector products, without ever forming or inverting Hessians. In benchmark CARA-Normal (Constant Absolute Risk Aversion with Gaussian distribution of uncertainty) environments, the approach recovers known analytical optima and converges reliably from random initialization. More broadly, because it is matrix-free, variance-reduced, and problem-agnostic, the framework extends naturally to complex nonlinear contracts where closed-form solutions are unavailable, such as sigmoidal wage schedules (logistic pay), relative-performance/tournament compensation with common shocks, multi-task contracts with vector actions and heterogeneous noise, and CARA-Poisson count models with $\mathbb{E}[X\mid a]=e^{a}$. This provides a new computational tool for contract design, enabling systematic study of models that have remained analytically intractable.
- North America > United States > Texas > Brazos County > College Station (0.04)
- North America > United States > Massachusetts > Suffolk County > Boston (0.04)
- Europe > Germany (0.04)
Modular and Integrated AI Control Framework across Fiber and Wireless Networks for 6G
Dzaferagic, Merim, Ruffini, Marco, Kilper, Daniel
The rapid evolution of communication networks towards 6G increasingly incorporates advanced AI-driven controls across various network segments to achieve intelligent, zero-touch operation. This paper proposes a comprehensive and modular framework for AI controllers, designed to be highly flexible and adaptable for use across both fiber optical and radio networks. Building on the principles established by the O-RAN Alliance for near-Real-Time RAN Intelligent Controllers (near-RT RICs), our framework extends this AI-driven control into the optical domain. Our approach addresses the critical need for a unified AI control framework across diverse network transport technologies and domains, enabling the development of intelligent, automated, and scalable 6G networks.
- Europe > Ireland > Leinster > County Dublin > Dublin (0.14)
- Oceania > Australia > New South Wales > Sydney (0.05)
- Information Technology > Communications > Networks (1.00)
- Information Technology > Artificial Intelligence (1.00)
AI-driven Java Performance Testing: Balancing Result Quality with Testing Time
Traini, Luca, Di Menna, Federico, Cortellessa, Vittorio
Performance testing aims at uncovering efficiency issues of software systems. In order to be both effective and practical, the design of a performance test must achieve a reasonable trade-off between result quality and testing time. This becomes particularly challenging in Java context, where the software undergoes a warm-up phase of execution, due to just-in-time compilation. During this phase, performance measurements are subject to severe fluctuations, which may adversely affect quality of performance test results. However, these approaches often provide suboptimal estimates of the warm-up phase, resulting in either insufficient or excessive warm-up iterations, which may degrade result quality or increase testing time. There is still a lack of consensus on how to properly address this problem. Here, we propose and study an AI-based framework to dynamically halt warm-up iterations at runtime. Specifically, our framework leverages recent advances in AI for Time Series Classification (TSC) to predict the end of the warm-up phase during test execution. We conduct experiments by training three different TSC models on half a million of measurement segments obtained from JMH microbenchmark executions. We find that our framework significantly improves the accuracy of the warm-up estimates provided by state-of-practice and state-of-the-art methods. This higher estimation accuracy results in a net improvement in either result quality or testing time for up to +35.3% of the microbenchmarks. Our study highlights that integrating AI to dynamically estimate the end of the warm-up phase can enhance the cost-effectiveness of Java performance testing.
- North America > United States > New York > New York County > New York City (0.05)
- North America > United States > California > Sacramento County > Sacramento (0.05)
- Europe > Italy > Abruzzo > L'Aquila Province > L'Aquila (0.04)
- (14 more...)
- Information Technology > Data Science > Data Mining (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.93)
- Information Technology > Artificial Intelligence > Machine Learning > Performance Analysis > Accuracy (0.68)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.67)
Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms
Xu, Shengwei, Zhang, Yichi, Resnick, Paul, Schoenebeck, Grant
Because high-quality data is like oxygen for AI systems, effectively eliciting information from crowdsourcing workers has become a first-order problem for developing high-performance machine learning algorithms. Two prevalent paradigms, spot-checking and peer prediction, enable the design of mechanisms to evaluate and incentivize high-quality data from human labelers. So far, at least three metrics have been proposed to compare the performances of these techniques [33, 8, 3]. However, different metrics lead to divergent and even contradictory results in various contexts. In this paper, we harmonize these divergent stories, showing that two of these metrics are actually the same within certain contexts and explain the divergence of the third. Moreover, we unify these different contexts by introducing \textit{Spot Check Equivalence}, which offers an interpretable metric for the effectiveness of a peer prediction mechanism. Finally, we present two approaches to compute spot check equivalence in various contexts, where simulation results verify the effectiveness of our proposed metric.
- North America > United States > New York > New York County > New York City (0.04)
- North America > United States > Texas > Travis County > Austin (0.04)
- North America > United States > Michigan (0.04)
- Europe > Slovenia > Central Slovenia > Municipality of Ljubljana > Ljubljana (0.04)
Arria NLG Appoints Mark Goodey to Lead Arria's Investment Analyst Business
Arria NLG, a leading provider of natural language generation (NLG) technologies, has appointed Managing Director and Innovation Strategist, Mark Goodey, to cement Arria Investment Analyst as the Banking, Financial Services, and Insurance (BFSI) industry leader. Arria Investment Analyst uses natural language technologies to bring 100 percent accuracy to investment analysis and to create data-driven investment commentary. "I am excited to lead this initiative," said Goodey. "Arria's Investment Analyst uses natural language technology to analyze investment portfolio performance. It's a technology uniquely placed to support asset managers, asset owners, and the financial services industry, so what used to take hours or days can now be accomplished in seconds."
Understanding Confusion Matrix
When we get the data, after data cleaning, pre-processing, and wrangling, the first step we do is to feed it to an outstanding model and of course, get output in probabilities. How in the hell can we measure the effectiveness of our model. Better the effectiveness, better the performance, and that is exactly what we want. And it is where the Confusion matrix comes into the limelight. Confusion Matrix is a performance measurement for machine learning classification.
The social life of Artificial Intelligence in education
Artificial intelligence is becoming a major feature of educational practice and policymaking, but researchers are beginning to raise critical questions about its ethics and effects. Artificial Intelligence (AI) has become the subject of both hype and horror in education. During the 2020 Covid-19 pandemic, AI in education (AIed) attracted serious investor interest, market speculation, and enthusiastic technofuturist predictions. At the same time, algorithms and statistical models were implicated in several major controversies over predictive grading based on historical performance data, raising serious questions about privileging data-driven assessment over teacher judgment. In the new special issue AI in education: Critical perspectives and alternative futures published in Learning, Media and Technology, Rebecca Eynon and I pulled together a collection of cutting edge social scientific analyses of AIed.
- Europe > United Kingdom (0.14)
- Asia > India (0.05)
- Information Technology (1.00)
- Government (1.00)
- Banking & Finance > Trading (0.88)
- (3 more...)
How to Evaluate Your Machine Learning Models with Python Code!
You've finally built your machine learning model to predict future prices of Bitcoin so that you can finally become a multi-billionaire. But how do you know that the model you created is any good? In this article, I'm going to talk about several ways you can evaluate your machine learning model with code provided! If you don't know the difference between regression and classification models, check out here. More specifically, I'm going to cover the following metrics: R Squared is a measurement that tells you to what extent the proportion of variance in the dependent variable is explained by the variance in the independent variables.
On improving learning capability of ELM and an application to brain-computer interface
Yayık, Apdullah, Kutlu, Yakup, Altan, Gökhan
As a type of pseudoinverse learning, extreme learning machine (ELM) is able to achieve high performances in a rapid pace on benchmark datasets. However, when it is applied to real life large data, decline related to low-convergence of singular value decomposition (SVD) method occurs. Our study aims to resolve this issue via replacing SVD with theoretically and empirically much efficient 5 number of methods: lower upper triangularization, Hessenberg decomposition, Schur decomposition, modified Gram Schmidt algorithm and Householder reflection. Comparisons were made on electroencephalography based brain-computer interface classification problem to decide which method is the most useful. Results of subject-based classifications suggested that if priority was given to training pace, Hessenberg decomposition method, whereas if priority was given to performances Householder reflection method should be preferred.
- Europe > Finland > Uusimaa > Helsinki (0.05)
- North America > United States > New Jersey (0.04)
- Asia > Middle East > Republic of Türkiye > Hatay Province > Iskenderun (0.04)
- Asia > Middle East > Republic of Türkiye > Ankara Province > Ankara (0.04)
- Health & Medicine > Therapeutic Area > Psychiatry/Psychology (1.00)
- Health & Medicine > Therapeutic Area > Neurology (1.00)